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Method and System for Content Aware and Energy Efficient 
Transmission of Videos and Images 

Field of The Invention 

[01] This invention relates generally to energy efficient transmission of 
multimedia data, and more particular to energy efficient transmission of 
layered video and images such as JPEG2000 video and JPEG2000 images. 

Background of the Invention 

[02] In general, wireless communications channels have a lower 
bandwidth and a higher bit error rate (BER) than wired channels due to severe 
channel conditions, such as path loss, fading, co-channel interference, and 
noise disturbances. Also, the throughput of the channels can fluctuate 
dynamically due to time varying characteristic of the channels. Overcoming 
the effects of the severe channel conditions is a major task in designing 
efficient transmission systems for multimedia, e.g., still images and videos. 

[03] Because multimedia tends to be highly redundant, it is preferred to 
apply compression to the source multimedia before transmission. The 
compressed multimedia has some special characteristics, such as unequal 
importance, error tolerance, and constrained error propagation. Unequal 
importance denotes that different parts of the compressed bitstream exhibits 
different perceptual importance. Error tolerance means that even if errors are 
introduced, the original information can still be reconstructed with minimal 
perceptual degradation. 
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[04] To improve the compression efficiency, variable length coding 
(VLC) is used by most prior art multimedia compression systems. However, 
VLC is very sensitive to unpredictable errors. If some bits are corrupted, then 
neighboring bits can also become useless. This is called error propagation. By 
applying error resilient coding encoding procedures, the propagation can be 
restricted inside a certain range. This is called constrained error propagation. 

[05] These three characteristics differentiate multimedia transmission 
from general voice, text and data communication. 

[06] Multimedia applications are becoming more common in wireless 
communication networks, such as cellular telephone networks, local area 
networks, and home networks. When compared to traditional text, voice and 
data, multimedia requires more bandwidth, and therefore, more transmission 
power. In addition, increasing the power can decrease the bit error rate. 

[07] However, more and more user devices are battery operated. 
Minimizing energy consumption for delivery of multimedia is important for 
such devices. 

[08] Energy consumption can be decreased by decreasing the 
complexity of encoders and decoders, by using low power circuitries, and by 
using low signaling-cost routing protocols. Network topologies can also be 
exploited to reduce energy consumption by using relay assisted transmission 
and power combining methods with diversity gain techniques. 
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[09] There is a trade-off between processing and transmission power 
consumption depending on the type and complexity of the multimedia, 
source and channel encoders in the transmitter, and source and channel 
decoders in the receiver. 

[010] A number of methods are known for energy efficient transmission. 
U.S. Patent Application 20030115428 of Zaccarin et al. June 19 2003 
describes a power management system that monitors a data buffer to 
determine appropriate processor clock speed or voltage. That allows a 
processor to switch to low power states whenever possible. That method 
does not address error rates and wireless transmission requirements. 

[01 1] U.S. Patent Application 20030103469 of Setty et al. June 5 2003 
describes a method for controlling transmission power in a time division 
duplex wireless telecommunication system. That method uses the size of the 
data and a midamble in a burst of data, and the change in rate matching to 
control the transmission power. However, that method also does not 
consider content, and only tries to maintain a predetermined SNR level for a 
minimum transmit power level. 

[012] U.S. Patent Application 20030101303 by Kung et al. describes a 
power-managing circuit for wireless communication. That circuit does not 
consider content characteristics. 

[013] U.S. Patent Application 20030100328 by Klein et al. May 8 2003 
describes a wireless local area network wherein mobile units receive beacon 
signals from access points. The access points control the power level of the 
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mobile units. They do not consider adaptation of encoding procedures, 
channel conditions, or distortion constraints. 

[014] U.S. Patent Application 20030086443 by Beach et al. describes a 
wireless data communication system for packet communications. A 
monitoring apparatus at an access point monitors all transmitted packets and 
packet arrival rates. Voice packets are sent immediately to a mobile unit, 
while other packets can be buffered at the access point. Packet arrival rates 
vary due to random delays. The packet arrival rate and delays are used to 
determine required power levels. 

[015] U.S. Patent Application 20030083088 by Chang et al. May 1 2003 
describes a wireless communications network that includes transmission 
power and data rate adaptation based on signal quality. They adapt power 
and data transmission rates. There is no consideration for allocating power 
according to distortion constraint of the content. 

[016] U.S. Patent Application 20030083036 by Liu et al May 1 , 2003 
describes a wireless transmission circuit with adjustable transmission power. 
The power level depends on a distance to a receiver. 

[018] U.S. Patent Application 20030064744 by Zhang et al. April 3, 
2003 describes a method for reducing power consumption in mobile devices. 
Their power allocation method maximizes a total effective data rate in the 
channel. 
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[019] Zhang et aL, "Power-Minimized Bit Allocation for Video 
Communications Over Wireless Channels," IEEE Trans. Circuits and 
Systems for Video Tech., v: 12, n: 6, 2002, describe a power allocation 
method that considers processing power for source encoding and channel 
encoding, as well as transmit power requirements. Their source coding 
method is strictly model-based. Their basic assumption is that one model 
works for all content. They also rely on an assumption that more complex 
source coding procedures achieve a lower bit rate. However, that is 
unrealistic in many cases. They also assume that the source processing 
power is decreased when the source rate is increased. That assumption 
cannot be generalized. They also erroneously assume that increasing the 
source rate requires more protection bits to satisfy distortion constraint. 
Those assumptions are due to the fact that their method is model-based. 
They consider complexity and energy consumption in a quantization 
process, but do not apply and consider error resilience source procedures and 
energy consumption with application of error resilience source encoding 
procedures. 

[020] Eisenberg et aL, "Joint Source Coding and Transmission Power 
Management for Energy Efficient Wireless Video Communications," IEEE 
Trans. Circuits and Systems for Video Tech., v: 12, n: 6, 2002, describe 
error resilience and concealment techniques at the source encoding level and 
transmission power management at the physical layer. They try to minimize 
overall transmission power. They couple expected distortion introduced by 
received packets only to source encoding parameters. That assumption 
neglects error propagation in the bit stream. Furthermore, their channel code 
and modulation rates are fixed. Their method operates off-line and is 
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computationally complex, and it is therefore not suitable for real-time 
applications. 

[021] Figure 1 shows the general features of prior art encoding systems. 
A joint source channel coding unit (JSCC) 150 receives a channel condition 
160 and constraints 140, e.g., delay or distortion. Based on these inputs and 
a rate distortion model 120 provided by a source encoder 110, the JSCC 
determines a source-encoding procedure 130 for the source encoder 110. 

[022] The source encoder receives multimedia 105 and applies the 
source-encoding to the multimedia to produce a compressed bit stream 115. 
A channel encoder 125 performs channel coding by adding error correction 
bits to the compressed stream and returns a protected bit stream 190. 

[023] Some prior art systems use a channel encoder that has a fixed 
channel code rate. Other systems apply different channel code rates on the 
fixed compressed bit stream depending on channel conditions and power 
constraints. This is called joint source channel matching with power control. 
In that case, the JSCC provides the channel encoder with a channel rate 135. 

[024] Prior art systems generally treat transmit power control and source- 
channel matching independently. Typically, the transmitter 170 allocates a 
transmit power level for a certain bit error rate at the receiver based on the 
channel condition 160. Then, the channel coding is matched to the source 
coding according to allocated power. Therefore, the transmitter does not 
provide input to the JSCC 150. The bit stream 180 is typically transmitted at 
a predetermined power level. 
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[025] Wei et al., in "Rate Efficient wireless Image Transmission using 
MIMO-OFDM," Unversity of Maryland, Institute of Systems Research, 
Technical Report, TR-2003-30, August 2003, describe how to use error 
resilient coding schemes during a source encoding stage to minimize error 
propagation. That scheme jointly allocates a source coding rate, source error 
resilient coding schemes, channel coding schemes, and channel coding rates. 
However, that scheme does not consider total energy consumption of the 
system during the allocation, nor does that scheme consider power levels in 
the transmitter. 

[026] The prior art does not efficiently optimize the transmit power level 
for multimedia based on characteristics of the multimedia, channel 
conditions, and complexities of source and channel encoding and decoding 
units. Nor does the prior art attempt to minimize total energy consumption 
while satisfying a distortion constraint. 

Summary of Invention 

[027] In the present invention, a quality scalable bitstream with multiple 
quality layers is generated in an optimal rate-distortion (R-D) sense from 
source multimedia. 

[028] Given an estimated channel condition, content, and an end-to-end 
rate-distortion constraint, the invention determines adaptively the number of 
layers to be transmitted, and adjusts the source encoding rate, the channel 
encoding rate and the transmit power level jointly for each layer. 



7 



MERL-1499 
Sahinoglu etal. 



Brief Description of the Drawings 

[029] Figure 1 is a block diagram of a prior art multimedia encoder; 

[030] Figure 2 is a block diagram of a multimedia encoder according to 
the invention; 

[03 1] Figure 3 is a block diagram of layered bit streams according to the 
invention; and 

[032] Figure 4 is a graph of an energy-distortion curve used by the 
invention. 

Detailed Description of the Preferred Embodiment 

[033] In a wireless communications network, transmission power is a 
major component of total energy consumption. Moreover, the energy 
consumption is proportional to the number of bits transmitted. Therefore, 
our invention minimizes energy consumption while meeting a predetermined 
quality of service (QoS) constraint for transmitting multimedia, e.g., still 
images, videos, voice, text, and data. 

[034] Our method selects an error resilient source encoder and a channel 
encoder according to dynamically varying channel conditions and signal to 
noise ratio (SNR) under a given rate-distortion constraint of the multimedia. 
The selected procedures and a selected transmit power level minimize total 
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energy consumption for delivery of the multimedia from a transmitter to a 
receiver. The total energy consumption is defined as the energy consumption 
due to processing and transmitting the multimedia. 

[035] As shown in Figure 2, we use an efficient joint source channel 
coding-power control (JSCC-PC) method and system 200. The method 
minimizes an objective function while satisfying a constraint based on 
energy and distortion. The objective function can minimize energy while 
meeting a minimum distortion requirement, or alternatively, the objective 
function can minimize distortion while meeting a minimum energy 
constraint. 

[036] From the source multimedia 205, the system 200 according to the 
invention generates a quality scalable bitstream 280, in a optimal rate- 
distortion (R-D) sense. 

[037] The bit stream 280 can include L layers 300, see Figure 3. The 
transmitter 200 includes a joint source channel coding and power control 
unit (JSCC-PC) 250, which uses rate-distortion characteristics 210 of the 
actual multimedia 205 to be transmitted. 

[038] In addition to descriptions 220 of a set of source error resilience 
procedures available to the source encoder 210. The system also considers 
constraints and objectives 280, channel condition 260, a channel codes for 
channel encoder 225, and power levels of a transmitter 270. The channel 
condition can include bandwidth, signal-to-noise ratio, and delay. 
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[039] In the preferred embodiment, the source encoding 210 is according 
to the JPEG 2000 standard, ISO/IEC, "ISO/IEC 15444-1:2000: Information 
technology - JPEG 2000 image coding system - part 1: core coding system," 
2000. However, it should be understood that other scaleable source encoders 
can also be used. 

[040] The channel encoder uses rate compatible punctured convolutional 
codes (RCPC), Hagenauer, "Rate-compatible punctured convolutional codes 
(RCPC) and their applications," IEEE Transactions on Communications, vol. 
36, no. 4, pp. 389-400, April 1988. To further improve the performance of the 
system, the transmitter power can vary over several levels, and can be 
adjusted dynamically by the system to meet current channel conditions. 

[041] As shown in Figure 3, each layer 300 of the encoded multimedia 
has a layer header 320 and a layer payload 330, with bits n H and n P 
respectively. An average distortion/bit of layer i is d i . An average error 
propagation per bit in layer i is b { . Therefore, each quality layer can be 
defined by a vector < d i ,b i ,n i > , where n. is a total number of bits after 
applying error resilience source encoding and unequal error protection in 
fields 310 and 340 of each layer 300 of the bit stream 299. 

[042] The selected error resilience source encoding procedure is applied 
to the multimedia to produce a particular layer that minimizes errors 
introduced by the wireless channel. There is a set of S source error resilience 
procedures. A procedure 5, e S is applied to layer i, where i = 1, . . ., L. The 
selected source encoding procedures are indicated by line 235. A difference 
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between values 235 and 130 is that the value 235 specifies selected source 
encoding procedures, while the value 130 that the source encoding 
procedure is fixed. 

[043] Each layer 300 is also protected by channel codes. There is a set of 
C of channel encoding procedures 230 that produce the error correcting 
codes for the channel encoder 225. A channel encoder C i 6 C is applied to 

layer z, where / = 1,. . L. The invention uses selected channel encoding 
procedures that can be applied to the layers. A difference between 240 and 
135 is that 240 specifies a selected set of channel coding procedures, while 
the value 130 specifies is a fixed procedure. 

[044] The transmitter 270 operates at several power levels, denoted by a 
set P 260. The set of possible transmit power levels for a layer i is denoted 
by P ( g P 250. A difference between 250 and 195 is that the value 250 
specifies selected power levels, while the value 195 is a fixed power level. 

[045] The energy required to transmit one bit at power level P { is e[ . The 

source encoding, channel encoding, and power level of each layer / can be 
specified by a vector < S i9 C i9 P t > . The energy consumption, due to 

computational complexities introduced by applying vector < S 0 C 0 P t > on 

layer i is as e c . We call this the processing energy consumption. This mainly 

takes place in three places: source encoding, channel encoding and baseband 
processing. For header protection, the energy consumption is determined by 
the code type and code rate, as well as the number of code words to be 
encoded. During decoding, the receiver end also consumes energy. Our 
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method also takes that into consideration, and receiver energy consumption 
is included in . Therefore, our method reduces energy for both the 

transmitter and the receiver. 

T 

[046] The vector for layer i is denoted with 1 for simplicity of the 
notation. The total energy consumption for processing, protecting and 

i Ei(Ti) = e c + n i e' i u 1 T 

transmitting layer i is 1 lJ « 1 1 , where i = 1, L. 

[047] After applying the source encoder, the channel encoder, and power 

level as specified in the vector 1 , the distortion per layer i is , for i = 
1, L. The JSCC-PC unit 250 selects the vector by minimizing an 
objective function and satisfying a constraint. The objective function can 
minimize overall energy consumption while satisfying the distortion 
constraint, or alternatively, the unit can minimize overall distortion while 
satisfying a energy constraint. The objective function and constraint can be 
formulated by 

MINXES) s.t. Z Dj(7i)>D ,and 

(JO 

MIN Z A (TO s.t. Z E^TJZE (i). 
(7i) /=i i=i 
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In the above, either the total energy consumption or distortion over L layers 
is minimized subject to a distortion or energy constraint, e.g., the overall 

L 

distortion ^ D t (T l ) must be lower than a distortion threshold D . 

/=i 



[048] The optimization-constraint problem given in Eq.l is solved with a 
convex hull analysis of an energy-distortion curve 400 as shown in Figure 4. 
First, the JSCC-PC unit 250 computes the resulting energy consumption 

E t (T x ) and the reduction in distortion G i (T x ) by applying vector T x 
=<S X ,C X ,P X > on layer i, where x = 1, ...,M, and* = 1, ...,L. Mis the 
number of vectors to consider. 



[049] The energy consumed when vector 7^ is applied onto layer i is 
determined. This is repeated for all M vectors. The resulting M energy 
values are reordered in the increasing order 
0<E i (T 1 )<E i (T 2 )<...<E i (T M ) 410. 



[050] The corresponding "reduction in distortion" values G, (r x ) 420 are 
also computed. Pairs of values (E i (T y ), G, (T y )) 430 that do not satisfy 

0 < G { (X x ) < G t (T 2 ) < . . . < G t (T M )) are discarded. The remaining M pairs 

440 are kept for further consideration. In other words, all the feasible 
solutions reside on the convex hull of the energy-distortion curve 450 for 
that layer. The same process is performed for each quality layer. 
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[051] After the feasible solutions for all the layers have been obtained, 
the optimal rate allocation and power control procedure for the optimization 
problem in equation (1) is solved as described below. 

[052] The following terminology is used 

AG, (s l ,s l ) = G l (s l )- G(s l ) : The distortion reductions by changing the vector 
for layer I from s l to s t . 

AEj (s, ,s l ) = E l (s l )- E(s l ) : The additional energy consumed by changing the 
vector for layer I from s t to s t . 

8i » s'i ) = ^i(Si,St) . formalized gain. 

G : Gain target to achieve to satisfy distortion constraint. 

[053] Before the process begins, the reduction in gain is initialized to 
zero, i.e., G=0. Then, the following steps are performed. 
For l</<Ldo 

Find feasible procedure sets 

Let s l = if, where 5° is a feasible procedure set with a lowest energy 
consumption, and mark s z ° . 
End for 
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While G<Gdo 

Find the layer I and the strategy s t , such that g,(j p ^) is maximized 
among all layers and all unmarked strategies for this layer. 
G = G + AG / (s / ,^); 

Set the vector for layer / to s l = s\ and mark s[ ; 
End while 

If G>G then 

Let / be a last layer with the vector s l * , adjust the length of this 
last layer to be -^(G-G^J/G,^) ; 

End if 

[054] Return the set of selected vectors for all the layers and the length 
of the layer. By adjusting the length of the last layer to be transmitted, the 
optimal solution can be approximated very precisely. 

[055] After selecting the vectors for each layers, the vector is applied to 
that layer, and the bit stream is generated. Each vector indicates the source, 
the source error resilience procedure, channel en coding procedure, the 
channel en coding rate and the transmit power level to be used for the 
corresponding layer. 

[056] Although the invention has been described by way of examples of 
preferred embodiments, it is to be understood that various other adaptations 
and modifications may be made within the spirit and scope of the invention. 
Therefore, it is the object of the appended claims to cover all such variations 
and modifications as come within the true spirit and scope of the invention. 
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